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ABSTRACT 

3 

This  Research  Memorandum  presents 
an  evaluation  of  past  efforts  to  esti¬ 
mate  the  savings  due  to  the  use  of  com¬ 
petition  in  weapon  system  production 
programs.  The  evaluation  was  based  on 
the  research  literature  available  from 
the  historical  efforts.  The  evaluation 
of  the  literature  discusses  in  detail 
the  analytical  models  and  techniques 
used  in  the  analyses  of  price  formation 
in  sole-source  and  competitive  produc¬ 
tion  programs,  the  data  bases  used  and 
reported  for  the  programs  studied,  and 
the  results  and  conclusions  reported  in 
the  literature.  The  memorandum  concludes 
that  the  models  and  techniques  used  to 
assess  the  price  effects  of  competition 
are  Immature  and  inadequate  to  sort  out 
the  effects  of  competition  sought  by  the 
analyses;  that  the  data  bases  used  in 
the  literature  are  incomplete,  of  uneven 
quality,  and  may  be  seriously  biased; 
and  that  the  resulting  estimates  of 
savings  due  to  the  use  of  competition 
are  of  uneven  validity. 
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INTRODUCTION 


Competition  has  been  the  policy  in  defense  procurement  for  a  number 
of  years.  In  fact,  competitive  procurement  was  one  of  the  improved  man¬ 
agement  practices  instituted  in  1962  as  part  of  a  5-year  program  for 
reducing  procurement  and  logistic  costs.  One  of  the  goals  of  this  pro¬ 
gram  was  to  have  40  percent  of  defense  procurement  done  competitively  by 
FY  1965  [21,  pp.  114-115]. 

The  DOD  policy  of  competition  is  based  primarily  on  a  belief  that 
competition  creates  cost  savings.  It  is  not  unusual  for  a  discussion  of 
the  effects  of  competition  to  include  a  statement  that  competition 
reduces  costs  by  "X”  percent.  The  value  and  base  of  the  "X"  percent 
vary  from  study  to  study.  A  study  done  by  the  Army  Procurement  Research 
Office  in  1978  notes  that  "Traditionally,  a  25  percent  reduction  is 
expected,  but  there  is  no  empirical  support  for  such  expectation" 

[3,  p.  1],  In  this  particular  instance  the  base  for  the  25  percent 
reduction  was  acquisition  costs. 

The  purpose  of  this  paper  is  to  evaluate  the  literature  dealing 
with  retrospective  estimates  of  savings  from  competition  in  production 
programs.  While  there  have  been  a  number  of  papers  with  the  same  pur¬ 
pose  (Beltramo  and  Jordan  [9],  Sherbrooke  [13],  and  Beltramo  [17]  are 
three  of  the  better  ones),  they  have  tended  to  focus  rather  narrowly  on 
specifics  such  as  data  manipulation  or  the  applicability  of  a  particular 
assumption.  This  paper  will  be  broader  and  deal  with  three  topics 
covered  in  the  procurement  literature.  The  first  topic,  which  receives 
cursory  treatment  In  the  other  reviews,  will  be  the  model  that  appears 
to  dominate  studies  of  defense  prices  in  both  sole-source  and  competi¬ 
tive  production  environments.  The  second  topic  will  be  the  statistical 
techniques  applied  to  the  analysis  and  their  appropriateness  for  fore¬ 
casting  the  effects  of  competition.  The  final  topic,  which  is  a  strong 
point  in  the  reviews  cited  above,  will  be  the  data  and  results  that 
appear  in  the  literature. 

MODELS 

Most  analyses  of  production  competition  focus  on  how  competition 
will  change  the  price  the  government  pays  for  a  weapon  system.  The 
focus  on  price  arises  from  the  fact  that  any  savings  that  result  from 
competition  are  derived  from  reductions  in  unit  procurement  prices  over 
the  life  of  the  system.  The  economic  question,  in  the  case  of  competi¬ 
tion,  is  whether  the  future  savings  are  large  enough  to  offset  those 
current,  nonrecurring  costs  of  implementing  competition  that  the  govern¬ 
ment  incurs  directly,  such  as  government-furnished  tooling  and  test 
equipment.  The  analyses  of  price  formation  and  competition  in  the 
literature  reviewed  here  attempt  to  answer  this  question. 

In  these  studies  the  models  address  the  unit  prices  to  the  govern¬ 
ment  under  sole-source  and  competitive  production  contracts.  This 
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comparison  yields  an  estimate  of  gross  savings  due  to  production  compe¬ 
tition.  Estimates  of  net  savings  are  obtained  by  subtracting  from  gross 
savings  the  nonrecurring  costs  to  the  government  of  establishing  the 
competition.  In  the  literature  considered  here,  these  nonrecurring 
costs  are  estimated  independently  of  the  price  analysis.  These  off-line 
estimates  are  then  subtracted  from  the  gross  savings  to  yield  net 
savings  due  to  competition. 

The  off-line  estimates  of  nonrecurring  cost  to  the  government  will 
be  addressed  in  the  data  section.  The  models  of  price  formation  used  in 
this  literature  are  addressed  in  this  section. 

The  Basic  Model 


Virtually  all  of  the  recurring  price/cost  models  in  defense  pro¬ 
curement  are  based  on  the  learning-curve  paradigm.  In  its  original 
form,  the  learning  curve  arose  from  an  observation  that  the  number  of 
manhours  required  to  produce  an  airframe  fell  with  an  increase  in  the 
total  number  of  airframes.  This  learning  effect  was  generalized  to  all 
inputs  and,  subsequently,  to  unit  production  costs. 

The  procurement  literature  tends  to  push  the  generalization  of  the 
learning  effect  a  step  further  by  assuming  (implicitly)  that  the  effects 
of  learning  are. pervasive  and  that  any  cost  reductions  are  passed 
through  directly  to  prices.  Given  this  implicit  assumption,  the  learn¬ 
ing  curve  is  transformed  into  a  price  improvement  curve  (PIC).1  The  PIC 
may  take  any  one  of  several  forms: 

•  A  simple  PIC  in  which  cumulative  quantity  is  the  only 
independent  variable 

•  A  rate-adjusted  PIC  that  includes  a  production-rate 
variable 

•  Other  forms  of  adjustment  to  the  simple  model  (see  Greer 
and  Liao  [16]  for  an  example  using  industry  capacity 
utilization  as  a  key  variable  influencing  price  behavior). 


1.  The  dominance  of  this  concept  is  evident  in  the  fact  that  [2,  3,  4, 
5,  8,  9,-  10,  12,  13,  16,  17,  20]  all  use  a  learning  curve  or  PIC-based 
approach.  Of  the  remaining  references,  only  [1]  is  concerned  with 
estimating  the  effects  of  competition  on  procurement  costs. 


The  typical,  rate  adjusted,  PIC  is  given  by: 


P  -  PjQ^h  ,  (1) 

where 

P  =■  price  of  the  Qth  unit 

P^  *  price  of  the  first  unit  produced  at  a  rate  of  one  per  period 
Q  =  cumulative  output 

R  -  production  rate  (quantity  per  period  of  time) 
a  *  learning  parameter 
b  =  rate  parameter. 

The  learning  and  rate  parameters,  a  and  b,  are  generally  trans¬ 
formed  into  percent  terms  in  specific  discussions  of  price  behavior. 

For  example,  a  value  for  a  of  -0.2  corresponds  to  an  87  percent  learn¬ 
ing  rate.  Assuming  a  constant  production  rate,  an  87  percent  learning 
rate,  in  the  PIC  context,  means  that  the  price  of  the  second  unit  is 
87  percent  of  the  price  of  the  first,  the  price- of  the  fourth  unit  is 

87  percent  of  the  price  of  the  second,  the  price  of  the  eighth  is 

87  percent  of  the  price  of  the  fourth,  etc.  The  rate  parameter  b  is 

interpreted  analogously. 

Given  the  basic  PIC  model  of  price  formation,  hypotheses  on  the 
effects  of  competition  are  generally  represented  as: 

•  A  shift  in  the  PIC,  i.e.,  a  change  (reduction)  in  P^ 

•  A  rotation  in  the  PIC,  i.e.,  a  change  (reduction)  in  a 

•  Both  a  shift  and  a  rotation  in  the  PIC. 

Figure  1  illustrates  these  effects  using  the  logarithmic  version  of  the 
PIC.  The  shift  is  represented  by  the  downward  displacement  of  the  curve 
and  the  rotation  by  the  increase  in  the  steepness.  These  shift-and- 
rotation  hypotheses  assume  that  the  structure  of  the  price  formation 
process  changes  with  the  introduction  of  competition.  Given  the 
increasing  emphasis  on  competition,  a  detailed  understanding  of  the 
assumed  structural  change  is  required  if  procurement  resources  are  to  be 
allocated  efficiently. 


1.  The  percent  learning  rate  (or  percent  slope)  is  defined  as  2a. 
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CUMULATIVE  QUANTITY 


FIG.  1:  THE  EFFECT  OF  COMPETITION  ON 
THE  ORIGINAL  SOURCE 


An  Evaluation  of  the  PIC  Methodology 

How  well  do  these  hypotheses  reflect  the  effects  of  competition  on 
the  prices  the  government  pays  for  goods  and  services?  This  question 
can  be  addressed  on  both  the  theoretical  and  empirical  levels.  The 
answer  depends  on  how  well  the  underlying  PIC  hypothesis  reflects  the 
price  formation  process  in  the  procurement  of  weapons.  Since  advocates 
of  competition  appeal  to  economic  theory  to  support  their  arguments,  the 
theoretical  foundations  of  the  PIC  will  be  considered  first. 

Theoretical  Evaluation 

If  an  economic  model  of  defense  procurement  prices  is  to  contribute 
to  an  understanding  of  the  effects  of  competition  on  weapons  system 
acquisition,  it  must  consider  the  institutional,  technical,  and  behav¬ 
ioral  characteristics  of  the  defense  procurement  process.  In  general, 
the  institutional  characteristics  (e.g.,  the  extent  of  competition  and 
the  Federal  Acquisition  Regulations),  and  the  technical  characteristics 
(e.g.,  the  type  of  product  being  purchased  (missile,  ship,  etc.)),  con¬ 
strain  the  behavior  of  the  participants  in  a  particular  acquisition. 

When  behavioral  assumptions  (e.g.,  firms  wish  to  maximize  their  profits) 
are  combined  with  the  institutional  and  technical  constraints,  a  model 
of  the  decision-making  process  and  its  effects  on  factors  of  interest, 
in  this  case  procurement  prices,  can  be  formed. 


The  PIC,  as  an  economic  model  of  procurement  prices  as  used  in  the 
literature,  is  too  narrowly  based.  It  lacks  sufficient  consideration  of 
the  characteristics  outlined  above  to  provide  even  a  minimal  understand¬ 
ing  of  the  price  formation  process.  In  particular  the  PIC  approach 
ignores,  with  one  exception  (see  the  discussion  of  [16]  below),  any 
consideration  of  the  behavioral  or  institutional  factors,  and  the  dis¬ 
cussion  of  the  technical  factors  contained  in  the  literature  (i.e., 
learning  and  rate  effects)  is  both  incomplete  and  confused. 

Behavior  and  Institutions.  Of  the  papers  examined  for  this  study, 
only  Greer  and  Liao  [16]  and  Beltramo  [17]  discuss  the  institutional/ 
behavioral  aspects  of  the  procurement  process.  Greer  and  Liao  approach 
the  behavioral  aspect  by  citing  three  alternative  strategies  a  firm  may 
choose  in  response  to  competition: 

•  Constant  percentage  profit — price  is  a  constant  percentage 
markup  over  cost. 

•  Penetration  (limit)  pricing — if  the  government  has  not 
committed  to  competition,  the  firm  sets  a  price  that  is 
low  enough  to  discourage  the  introduction  of  competition. 

•  Skimming  pricing — if  the  government  is  committed  to  compe¬ 
tition,  the  firm  sets  a  high  price  and  lowers  it  as  neces¬ 
sary  to  meet  competition. 

Greer  and  Liao  never  follow  up  explicitly  on  these  particular  alterna¬ 
tives.  However,  their  empirical  work  does  include  a  measure,  industry 
capacity  utilization,  that  reflects  some  consideration  of  the  behavioral 
alternatives  available  to  a  firm. 

Beltramo  [17,  pp.  40-41]  notes  that  a  variety  of  economic  models 
can  be  used  to  describe  the  behavior  of  a  firm  and  that  the  particular 
model  depends  upon  the  situation.  Beltramo  suggests  that  the  approach 
found  in  the  Stackelberg  duopoly  model  might  be  appropriate  for  dealing 
with  dual-source  competition. 

The  Stackelberg  model  recognizes  two  types  of  firms.  A  firm  may 
choose  to  be  a  leader  and  pursue  a  dominant  market  position  by  using 
aggressive  strategies  in  areas  such  as  price  and  quality.  Alterna¬ 
tively,  the  firm  may  choose  to  be  a  follower  and  adopt  a  set  of  passive 
strategies  in  areas  of  potential  competition.  In  this  instance  the  fol¬ 
lower  firm  will  serve  that  segment  of  the  market  that  the  leader  firm 
cannot  supply,  possibly  ending  up  with  a  small  but  profitable  market 
niche. 

1.  If  a  contractor  knows  the  government's  criteria  for  introducing 
competition,  the  limit  price  can  be  calculated  and  the  profitability  of 
this  approach  compared  to  that  of  other  pricing  strategies. 


Although  the  loser  in  a  dual-source  competition  may  receive  a  zero 
award,  it  is  not  a  common  occurrence,  and  often  the  loser  is  guaranteed 
sofae  part  of  the  fiscal-year  buy.  This  quantity  guarantee  can  provide  a 
substantial  incentive  for  passive  pricing  behavior  by  the  firms  in¬ 
volved.  Only  when  both  firms  desire  to  win  the  competition  and  choose 
aggressive  pricing  strategies  can  the  government  have  a  reasonable 
expectation  of  financial  benefits  from  dual-source  competition. 

While  Beltramo's  discussion  is  an  excellent  beginning  toward  devel¬ 
opment  of  a  complete  model  of  pricing  behavior,  the  determinants  of 
pricing  strategy  and  the  implications  for  empirical  models  are  not 
explored.  Ultimately,  Beltramo  relies  on  the  PIC  formulation  for 
empirical  work. 

Technical  Considerations .  The  PIC  model,  as  found  in  the  litera¬ 
ture,  is  essentially  a  cost-based  model  of  price.  However,  the  only 
technical  factors  included  are  the  production  rate  and  the  cumulative 
quantity.  The  exclusion  of  other  relevant  variables,  such  as  resource 
prices  and  system  characteristics,  suggests  that  the  PIC  is  an  incom¬ 
plete  model  of  prices.  This  conclusion  is  reinforced  by  the  universal 
failure  of  the  literature  to  explain  how  the  firms  transform  their  costs 
into  price  offers. 

The  incomplete  specification  of  price  formation  represented  by  the 
PIC  is  a  critical  problem  for  two  reasons.  First,  the  failure  of  the 
literature  to  include  a  number  of  potentially  important  factors  in  the 
model,  factors  that  are  recognized  as  important  at  other  levels  of  anal¬ 
ysis,  implies  a  fundamental  conceptual  incompleteness  in  the  treatment  ' 
of  both  price  formation  in  general  and  competition  in  particular.  The 
danger  is  that  the  simple  model  will  result  in  simple  and  inappropriate 
solutions  to  a  complex  problem. 

Second,  the  exclusion  of  important  variables  from  the  analysis 
implies  that  the  empirical  effects  of  competition  on  prices  may  be  esti¬ 
mated  with  systematic,  but  unknown,  errors.  This  statistical  problem 
compounds  the  conceptual  problem  cited  above. 

The  fundamental  problems  associated  with  an  incomplete  specifica¬ 
tion  are  compounded  by  the  widespread  confusion  over  what  the  rate  term 
is  supposed  to  represent.  Most  discussions  of  the  effect  of  the  produc¬ 
tion  rate  on  price  focus  on  the  cost-based  concept  of  economies  of 
scale.  Economies  of  scale  are  an  important  consideration  in  planning 
the  optimal  level  of  tooling,  but  they  are  not  necessarily  relevant  in 
an  evaluation  of  rate  effects  on  prices  in  the  short  run.  In  terms  of 
the  effect  on  cost,  the  question  to  be  considered  in  the  economies-of- 
scale  context  is  the  rate  at  which  a  facility  is  designed  to  produce, 
i.e.,  actual  scale,  relative  to  the  optimal  scale. 

1.  One  factor  that  may  reduce  the  Importance  of  the  economies-of-scale 
issue  is  that  the  optimal  scale  may  not  be  unique,  i.e.,  there  may  be  a 
range  of  facility  sizes  that  produce  at  equivalent  average  unit  costs. 


To  the  extent  that  cost  variations  are  an  important  influence  on  a 
firm's  pricing,  the  relevant  production  rate  concept  is  one  associated 
with  the  operating  costs  of  the  producer.  In  this  context  a  measure  of 
the  production  rate  relative  to  the  design  rate  or  plant  size,  not  the 
absolute  production  rate,  would  appear  to  be  a  better  approach  for  esti¬ 
mating  rate  effects.  The  importance  of  this  factor  depends  on  the  tech¬ 
nology  of  the  process  used  in  manufacturing  the  particular  product  and 
the  planning  horizon  used  in  the  analysis. 

As  noted  above,  the  PIC  models  do  not  explicitly  address  the  behav¬ 
ioral  or  institutional  aspects  of  pricing,  and  they  incompletely  address 
the  technical  considerations.  Consequently,  the  application  of  the  PIC, 
given  by  equation  1,  implicitly  assumes  that  the  only  factors  important 
to  the  price  are  the  costs  and  that  these  costs  are  passed  through 
directly  to  the  government.  The  shift  and/or  rotation  of  the  PIC  when 
competition  is  introduced  amounts  to  an  assumption  that  competition 
causes  firms  to  become  technically  smarter,  more  efficient,  or  less  con¬ 
cerned  with  profits,  or  some  combination  of  these  factors.  An  adequate 
understanding  of  the  potential  benefits  of  competition  is  hindered  by 
the  failure  of  the  PIC  methodology  to  address  the  mechanism  that  causes 
the  firms  to  change  their  behavior. 

Stability.  The  fundamental  empirical  problem  with  the  PIC  approach 
to  competition  analysis  is  that  the  learning  curves  that  form  the  foun¬ 
dation  of  this  price  analysis  tend  to  be  highly  unstable.  The  evidence 
on  stability  comes  from  work  done  by  Alchian  [22]  during  World  War  II 
but  not  available  for  public  release  until  1963.  Alchian  estimated 
learning  curves  for  the  production  of  World  War  II  military  airframes 
using  direct  labor  hours  per  pound  of  airframe  as  the  independent 
variable. 

The  principal  purpose  of  the  research  was  to  evaluate  the  learning 
curve  as  an  analytical  tool  by  examining  the  stability  of  the  learning 
phenomenon.  In  this  instance,  stability  was  defined  in  terms  of  the 
ability  of  the  model  to  predict  the  cumulative  manhours  required  to  pro¬ 
duce  the  1000th  airframe,  and  the  best  model  was  defined  as  the  one  with 
the  lowest  mean  absolute  error  of  prediction.  Alchian 's  best  model  had 
a  mean  absolute  prediction  error  of  22  percent  with  a  range  of  errors 
between  -44  percent  and  +116  percent  [22,  p.  690].  In  addition,  this 
best  model  used  the  initial  program-specific  data,  rather  than  histori¬ 
cal  data  from  other  programs,  to  define  the  parameters  of  the  learning 
curve • 

Since  price  analysis  in  the  literature  has  generally  been  done  on  a 
program-by-program  basis,  the  results  cited  above  have  important  impli¬ 
cations  for  the  degree  of  confidence  that  should  accompany  estimates  of 
sole-source  and  competitive  program  costs.  The  instability  of  the 
underlying  relationship  is  compounded  by  the  lack  of  consideration  for 
the  linkages  among  production  inputs,  costs,  and  prices.  This  evidence 
on  the  instability  of  the  basic  learning  curve  reinforces  the  view  that 


the  simple  PIC-based  analysis  Ignores  too  many  factors  to  be  of  much  use 
In  evaluating  the  effects  of  competition  on  procurement. 

Survivability.  Despite  these  serious  problems,  the  learning-curve 
based  PIC  has  endured  in  some  aspects  of  DOD/DON  price  analysis,  such  as 
GFE.  This  survivability  reflects  the  perception  that  defense  procure¬ 
ment  prices  tend  to  fit  the  implied  learning-curve  pattern.  This  is 
particularly  true  of  sole-source  programs,  and  the  rate  of  learning  is 
often  a  point  of  discussion  in  negotiations  over  sole-source  prices. 

There  are  a  number  of  theoretical  considerations  that  would  tend, 
in  a  stable  environment,  to  reinforce  a  firm's  incentives  to  establish 
the  PIC  pricing  pattern.  First,  the  observed  pattern  of  prices  may 
reflect  a  case  of  self-fullfilling  expectations.  If  the  government 
accepts  the  PIC  as  the  way  prices  should  behave,  a  firm  can  minimize 
negotiation  costs  and  reduce  the  potential  for  other  problems  by  having 
prices  behave  in  the  expected  manner. 

Second,  there  is  the  question  of  contractor  performance.  In  the 
early  stages  of  a  program  the  contractor  may  not  know  if  the  system  can 
be  delivered.  This  risk  of  nonperformance  may  be  reduced  with  more 
resources,  i.e.,  a  higher  price.  As  the  program  progresses  success¬ 
fully,  both  the  risk  of  nonperformance  and  the  price  may  fall. 

Third,  there  is  the  question  of  government  performance.  Since  pro¬ 
grams  are  typically  funded  by  Congress  one  year  at  a  time,  there  is  the 
risk  that  the  government  will  cancel  or  substantially  alter  the  program. 
An  example  of  such  an  alteration  is  the  program  stretchout. 

The  government  performance  risk  can  influence  the  firm  in  two  ways. 
First,  there  are  set-up  costs  associated  with  establishing  a  program, 
and  these  costs  are  independent  of  the  quantities  produced.  Given  the 
risk  of  program  cancellation,  the  firm  has  an  incentive  to  charge  a 
higher  price  early  in  the  program  in  an  attempt  to  cover  these  set-up 
cost s •  As  the  program  continues,  the  unrecovered  portion  of  these  costs 
and  the  associated  risk  falls.  Second,  since  materials  used  in  many 
weapons  systems  must  be  ordered  well  before  they  are  actually  required, 
the  contractor  must  obtain  resources  based  on  the  proposed  program.  A 
program  cancellation  or  stretchout  leaves  the  contractor  with  higher 
costs  than  if  the  program  were  operated  as  planned,  again  providing  an 
incentive  for  a  high  initial  price.  As  the  program  progresses,  the 
plans  are  likely  to  become  more  stable,  reducing  the  risk  of  cancella¬ 
tion.  This  progressive  reduction  in  risk  may  be  reflected  in  progres¬ 
sively  lower  prices. 

When  the  risk  factors  are  combined  with  the  government's  expected 
price  profile,  the  corporate  negotiators  have  strong  incentives  to 
establish  high  first  prices,  which  then  decline  with  cumulative  produc¬ 
tion.  The  important  factor  here  is  that  the  declining  price  profile  may 


be  due  as  much,  or  more,  to  the  government's  way  of  doing  business  as  it 
is  due  to  learning  on  the  part  of  the  contractor. 


Prediction.  Another  important  characteristic  of  the  PIC  approach 
used  in  the  literature  is  the  fact  that  most  analysis  and  estimation  are 
performed  on  a  program-by-program  basis.  Consequently,  on  any  particu¬ 
lar  program,  the  data  base  consists  of  a  small  number  of  annual  observa¬ 
tions1  that  are  the  outcomes  of  a  sequence  of  decisions  made  in  a 
relatively  stable  environment.  It  is  common  to  find  a  strong  correla¬ 
tion  between  sets  of  time-series  observations,  whether  or  not  there  is  a 
good  theoretical  explanation  for  the  relation. 

Even  so,  it  is  Important  to  keep  in  mind  that  the  predictive  accu¬ 
racy  of  the  PIC  (such  as  it  is)  only  holds  within  a  single  program. 

This  accuracy  pertains  to  predicting  the  prices  of  later  buys  given  the 
prices  of  earlier  buys.  There  is  no  ability  at  all  to  predict  the 
earlier  prices  themselves.  As  is  argued  in  the  next  section,  it  is  cru¬ 
cial  to  be  able  to  predict  these  early  prices,  since  competition  may 
change  all  prices. 

Structural  Change.  The  PIC  analysis  in  the  literature  is  based  on 
data  drawn  from  an  era  in  which  competition  was  used  early  in  the  pro¬ 
gram  life  cycle — such  as  a  sustained  concept  or  design  competition 
between  two  or  more  contractors.  Competition  for  full-scale  engineering 
development  (FSED)  was  common,  but  competition  between  two  sources 
during  FSED  was  not  common.  Dual  source  production  competition  was  used 
only  for  exceptionally  strong  reasons.  The  introduction  of  competition 
as  the  baseline  approach  to  production  procurement  represents  a  major 
change  in  the  way  DOD  does  business.  If  firms  change  the  way  they  price 
their  products,  which  should  be  expected  since  firm  pricing  decisions 
are  based  on  the  business  environment  in  which  the  firm  operates,  then 
the  simplistic  PIC  approach  taken  in  the  literature  will  be  an  even  less 
adequate  base  for  analysis. 

A  prime  example  of  this  structural  change  phenomenon  can  be  drawn 
from  the  economics  literature.  From  the  late  1950s  until  the  mid-1970s 
the  Phillips  Curve  was  one  of  the  principal  conceptual  and  empirical 
tools  for  macroeconomic  policy  makers.  The  Phillips  Curve  was  based  on 
the  observation  that  high  rates  of  inflation  and  high  rates  of  unemploy¬ 
ment  never  occurred  simultaneously.  Consequently,  by  following  an 
inflationary  policy,  government  could  keep  unemployment  at  low  levels. 
However,  in  1975  the  inflation  rate  and  unemployment  rate  both  hovered 
around  7  percent,  an  impossible  combination  given  the  conventional  wis¬ 
dom  of  the  Phillips  Curve.  Subsequent  research  into  the  phenomenon 
found  that  the  supposed  tradeoff  was  only  valid  for  random  variations  of 


1.  A  typical  number  of  observations  on  a  program  would  be  five  or  six, 
so  ten  would  appear  to  be  a  large  number  of  observations  for  this 
approach  to  price  analysis. 


inflation  around  a  stable  mean,  and  that  the  advent  of  systematic 
increases  in  the  inflation  rate  changed  the  relationship  substantially. 

The  acquisition  community  is  likely  to  experience  considerable  dif¬ 
ficulty  in  evaluating  prices  in  general,  and  the  effects  of  competition 
in  particular,  during  the  transition  to  the  new  way  of  doing  business. 
This  difficulty  is  likely  to  persist  until  the  procurement  environment 
stabilizes  and  both  firms  and  government  have  had  an  opportunity  to  sort 
out  the  available  information  and  determine  appropriate  rules  of 
behavior. 

The  Anticipation  Hypothesis 

The  discussion  above  concentrated  on  general  problems  associated 
with  the  PIC  approach  to  price  analysis  and  the  potential  problems  in 
applying  the  PIC  to  a  competitive  environment.  Even  if  one  discounts 
the  general  considerations  outlined  above,  there  are  specific  problems 
with  any  application  of  the  shif t-and-rotation  model.  The  best  way  to 
illustrate  the  problems  is  to  present  an  alternative  hypothesis  of  the 
behavior  of  a  first  source  faced  with  the  possibility  of  competition. 
This  hypothesis  states  that,  in  anticipation  of  competition,  the  first 
source  increases  its  initial  price  and  reduces  the  rate  of  price 
improvement  during  the  period  of  sole-source  purchases.  By  responding 
to  the  threat  of  competition  in  this  manner,  the  first  source  increases 
the  current  flow  of  profits  in  response  to  the  increased  risk  of  lost 
revenues  and  profits  due  to  the  introduction  of  the  second  source. 

The  anticipation  hypothesis  is  particularly  important  (and  damaging 
to  the  standard  methodology)  because  it  has  the  same  implications  for 
the  change  in  the  parameters  of  the  PIC  as  the  shif t-and-rotation 
hypothesis,  i.e.,  a  decrease  in  the  first  unit  price  and  an  increase  in 
the  learning  rate  after  the  introduction  of  competition.  Consequently, 
no  empirical  evaluation  of  competition  based  on  the  PIC  formulation  can 
distinguish  between  the  shift  and  rotation  hypothesis  and  the  anticipa¬ 
tion  hypothesis.  The  conclusions  on  the  effects  of  competition  depend 
crucially  on  which  hypothesis  is  correct.  Acceptance  of  the  shif t-and- 
rotation  hypothesis  when  the  anticipation  hypothesis  is  correct  will 
result  in  overestimation  of  the  benefits  of  competition. 

The  overestimation  results  from  the  fact  that  under  the  shift-and- 
rotation  hypothesis  the  entire  downward  movement  of  the  PIC  is  credited 
to  competition.  However,  if  the  anticipation  hypothesis  is  correct,  the 
observed  movement  of  the  PIC  should  be  decomposed  into  two  parts. 

First,  there  would  be  an  upward  movement  of  the  PIC  as  firms  anticipated 
competition,  and  second,  a  downward  shift  as  competition  becomes  effec¬ 
tive.  The  increased  costs  associated  with  the  upward  shift  should  be 
subtracted  from  the  reduced  costs  observed  when  competition  becomes 
effective. 


Summary 


Three  points  need  to  be  made  with  respect  to  the  PIC  model.  Point 
one  is  that  the  PIC,  as  applied  in  the  literature,  has  minimal  theoreti¬ 
cal  content.  The  implicit  assumption  of  the  PIC  approach  is  that,  given 
an  initial  price,  only  learning  and  production  rate  effects  are  impor¬ 
tant  in  determining  price.  This  assumption  is  maintained  in  spite  of  the 
recognition  of  other  factors  that  are  believed  to  influence  cost  and 
price. 


Point  two  is  that  the  ability  of  the  PIC  to  predict  prices  depends 
on  the  stability  of  the  procurement  environment.  If  the  only  variable 
in  a  program  is  the  production  rate  (and  consequently,  cumulative  out¬ 
put),  the  PIC  may  prove  adequate  for  predicting  future  program  costs 
once  initial  values  are  established.  However,  the  introduction  of  com¬ 
petition  as  the  rule  in  procurement  may  represent  a  fundamental  change 
in  the  business  environment.  If  so,  firms  can  be  expected  to  change  the 
way  they  price  their  products,  and  the  PIC  approach,  unless  modified, 
will  be  inapplicable,  since  it  is  based  on  Information  drawn  from  a 
different  environment. 

Point  three  is,  from  a  practical  view,  the  most  damaging  to  the  PIC 
approach  to  estimating  the  effects  of  competition.  It  is  simply  that 
there  is  an  alternative  hypothesis,  the  anticipation  (skimming)  hypoth¬ 
esis,  with  the  same  measured  effects  on  the  PIC  as  the  shift-and- 
rotation  hypotheses  and  substantially  different  conclusions  about  the 
cost  effectiveness  of  competition.  In  particular,  savings  calculated 
under  the  shif t-and-rotation  hypotheses  will  overestimate  the  benefits 
of  competition  if  the  anticipation  hypothesis  is  correct. 

TECHNIQUES 

Estimating  Equations 

While  equation  1  represents  the  basic  model  of  the  price  improve¬ 
ment  curve  (PIC),  it  does  not  represent  the  estimating  equation  that  is 
typically  used  in  evaluating  the  behavior  of  prices  in  specific  cases. 
The  application  of  equation  1  would  require  detailed  observations  on  the 
price  charged  for  each  unit  of  the  product.  The  information  that  is 
typically  available  consists  of  the  number  of  units  purchased  in  a  given 
year  and  the  total  cost  of  that  particular  buy. 

One  of  the  techniques  used  in  the  literature,  and  the  one  most 
widely  used,  is  to  estimate  a  log-log  form  of  equation  1  as  given  by: 


log(p)  -  log(P1)  +  b[log(R) ]  +  a [ log(Qm) ] 


(2) 


where  the  unit  of  observation  is  the  annual  buy,  p  is  the  average 
price  of  the  units  in  the  buy,  R  is  the  lot  (buy)  size,  and  Qm  is 


the  lot  mid-point.  Qm  is  calculated  as  the  sum  of  all  units  produced 
prior  to  the  current  lot  plus  half  the  current  lot. 

One  criticism  of  the  log-linear  technique  is  directed  at  the  us^  of 
the  lot  mid-point,  Qm.  Given  the  observed  average  lot  t  price,  P  , 
the  true  lot  t  mid-point  value,  Qt»  is  given  by: 


which  in  general  does  not  equal  Qm  ( “Qt-i  +  R^/2  ) .  The  consequence  of 
this  measurement  error  is  that  the  values  of  the  price  improvement 
parameter,  a,  and  the  production  rate  parameter,  b,  are  estimated 
with  systematic  but  unknown  errors.  These  errors  will  be  carried 
through  in  the  evaluation  of  both  sole-source  and  competitive  pricing 
behavior. 

A  second  technique  for  estimating  the  parameters  of  the  PIC  has 
been  used  [29].  The  estimating  equation,  which  avoids  the  measurement 
problems  of  the  leg-linear  approach,  is  derived  by  integrating  equa¬ 
tion  1.  The  equation  is  given  by: 

TLCC  -  P^  [qj+a  -  ]  /d  +  a)  .  (4) 

where 

TLC  ■  total  cost  of  lot  t 
P^  ■  first  unit  price 

R  ■  production  rate  proxy,  i.e.,  size  of  lot  t 
Qt  ■  cumulative  output  through  lot  t 

Qt_^  ■  cumulative  output  through  lot  t  -  1 
a  “  learning  parameter 
b  ■  rate  parameter. 

The  estimation  procedure  uses  a  complex  iterative  procedure  to  find 
the  values  of  P^,  a,  and  b  that  minimize  the  sum  of  squared  errors. 
Even  if  the  underlying  problems  of  the  PIC  model  are  ignored,  there  is 


still  a  fundamental  problem  with  this  complex  iterative  approach.  The 
properties  of  the  estimators  from  the  procedure  are  unknown  for  small 
samples*  As  noted  previously,  in  attempting  to  estimate  effects  of  com¬ 
petition,  a  large  sample  would  be  10  observations.  However,  a  sample  of 
10  observations  is  considered  to  be  small  for  purposes  of  estimation. 
Consequently,  there  is  no  applicable  statistical  technique  for  express¬ 
ing  the  degree  of  confidence  in  the  estimates. 

How  the  PIC  Is  Used  To  Estimate  Cost  Savings 

There  are  several  detailed  descriptions  in  the  literature  of  how 
cost  savings  are  estimated  from  a  PIC  (see,  for  example,  APR078  [3, 
pp.  8-16]).  The  general  procedure  is  illustrated  in  figure  2.  A  PIC  is 
fitted  to  the  observations  on  price  before  competition  is  introduced — 
shown  as  the  solid  line.  The  estimated  PIC  is  then  used  to  forecast  the 
prices  that  would  have  been  charged  if  the  program  had  remained  sole 
source.  These  forecast  prices  are  shown  as  the  dashed  line.  Savings  in 
recurring  costs,  by  lot,  are  estimated  as  the  differences  between  the 
forecast  prices  and  the  post-competition  observations. 


CUMULATIVE  QUANTITY 

FIG  &  THE  PIC  AND  ESTIMATED  SAVINGS 


Given  the  estimated  savings  for  each  lot,  the  procedure  for  calcu¬ 
lating  total  program  savings  varies  depending  on  the  availability  of 
other  data  and  on  the  judgement  of  the  researcher.  For  example,  if  data 
on  nonrecurring  costs  are  not  available,  then  the  sum  of  the  individual 
lot  savings  may  be  presented  as  the  estimate  of  gross  savings.  If  non¬ 
recurring  costs  are  available,  then  the  researcher  must  decide  whether 
to  discount  the  savings  to  account  for  the  time-value  of  money  and  what 
discount  rate  to  use.  Whether  discounted  or  not,  the  result  will  be 
presented  as  net  savings. 

One  of  the  consequences  of  the  small  sample  sizes  common  in  the 
literature  is  an  inability  to  estimate  a  PIC  from  the  available  data. 

As  a  consequence,  a  price-improvement  parameter  or  production-rate 
parameter  often  is  imposed  on  the  data  by  the  researcher(s) .  The  impo¬ 
sition  of  an  incorrect  value  for  either  of  these  parameters  can  have  a 
substantial  effect  on  the  estimated  savings  from  competition. 

The  effect  of  imposing  too  high  a  rate  of  price  improvement  is 
illustrated  in  figure  3.  The  imposed  PIC  is  illustrated  by  the  solid 
line.  The  estimate  of  savings  is  definitely  low  compared  to  the  savings 
that  were  realized  relative  to  the  "true"  PIC,  illustrated  by  the  dashed 
curve . 


CUUULATJVt  QUANTITY 


FIG  3:  TOO  HIGH  A  RATE 
OF  PRICE  IMPROVEMENT 


-14- 


The  discussion  above  has  presented  the  theoretical  and  empirical 
underpinnings  of  the  PIC  approach  to  price  analysis  and  the  effects  of 
competition.  The  remainder  of  the  paper  will  consider  the  characteris¬ 
tics  of  the  data  base  used  in  estimating  the  effects  of  competition  and 
the  robustness  of  the  resulting  estimates  of  the  effects  of  competition. 

DATA  BASE 

This  section  describes  a  typical  sample  in  the  literature,  indi¬ 
cates  which  cases  have  been  analyzed,  and  identifies  variables  used  in 
the  analyses. 

The  Sample 

A  typical  sample  consists  of  the  different  production  lots  within  a 
single  program.  As  a  consequence,  samples  tend  to  be  small  (usually 
with  fewer  than  ten  observations). 

Figure  4  shows  the  data  from  the  AGM-12B  missile  program  for  the 
original  producer  (Martin).  An  important  feature  is  the  paucity  of 
data,  with  only  three  data  points  before  competition  and  six  after.  The 
AGM-12B  program  is  typical  in  this  respect. 


L^CempetMon  Introduced 


10000 


CUMULATIVE  QUANTITY 


100000 


FIG.  A:  PRICE  IMPROVEMENT  CURVE  FOR  TEE 
AGM-12B  MISSILE  MANUFACTURED  BY  MARTIN 
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Small  samples  make  statistical  estimation  difficult,  and  the 
resulting  estimates  have  large  variances.  The  AGM-12B  program  is  a  case 
in  point.  Although  Martin  appears  to  have  changed  pricing  behavior 
after  competition  was  introduced,  the  data  fail  to  show  statistically 
significant  differences.  In  this  instance,  the  availability  of  only 
three  data  points  before  the  beginning  of  competition  did  not  provide 
sufficient  information  for  comparison  to  the  competitive  situation. 

Cases 


The  literature  review  touched  on  53  cases  analyzed  in  13  different 
studies.  One  study  typically  covered  several  programs;  however,  each 
program  was  analyzed  separately,  without  comparisons  across  programs. 

Table  1  displays  some  of  the  characteristics  of  these  cases.  The 
subheadings  in  the  first  column  categorize  the  items  being  procured: 
munitions,  missiles,  or  electronics.  The  second  column  describes  the 
type  of  competition:  winner-take-all  or  split  buy.  The  third  column 
lists  the  years  of  production. 

To  summarize,  of  the  53  cases,  there  were  30  electronic  subsystem 
or  component  programs,  13  missile  or  missile  parts  programs,  5  bomb  or 
munitions  programs,  and  5  torpedo  programs. 

Over  half  of  the  programs  (33)  had  used  the  winner-take-all  compet¬ 
itive  method,  in  which  the  entire  production  quantity  is  awarded  to  the 
winner.  Three  programs  divided  the  production  between  two  or  more  com¬ 
petitors  in  what  are  called  dual-source  or  split  buys.  Twelve  programs 
were  competitive  split  buys  in  some  years,  winner-take-all  in  others, 
usually  a  buyout  after  the  competitive  split  buy.  The  procurement 
methods  for  five  programs  were  not  identified. 

Selectivity  Bias 

Several  aspects  of  the  cases  used  in  this  literature  suggest  the 
possibility  of  selectivity  bias.  That  is,  the  cases  may  not  be  typical 
of  what  can  be  expected  from  competition  in  the  future,  particularly  the 
results  of  cost  savings. 

•  The  PIC  methodology  that  is  standard  in  this  literature 
for  estimating  competitive  cost  savings  can  use  only  cases 
that  are  sole  source  initially  and  then  competitive. 

Thus,  production  programs  that  were  competitive  from  the 
outset  were  excluded. 


CASE  STUDIES  AND  RESULTS  OF  COMPETITION 
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The  time  period  for  which  information  is  available  to  allow  classification  by  type  of  competition. 
No  information  is  available  to  confirm  Beltramo's  classification.  Some  of  these  cases  are  from  [1 
which  the  study  team  was  unable  to  obtain. 


•  That  the  cases  were  first  sole  source,  then  competitive, 
raises  the  possibility  that  some  of  the  competitions  may 
have  been  used  as  a  means  of  solving  problems  with  sole- 
source  programs.  A  sole-source  program  with  problems  may 
have  been  a  high-cost  program  as  well. 

Without  these  problems,  competition  might  have  a  much 
lower  payoff.  As  Beltramo  notes,  "A  sole-source  procure¬ 
ment  that  is  being  kept  'on  track'  by  an  innovative,  well- 
motivated  contractor  with  a  steep  learning  curve  may  not 
be  affected  by  the  introduction  of  a  competitive 
procurement.” 

Another  indication  that  competition  might  be  a  pro¬ 
duct  of  poor  sole-source  performance  is  discussed  by 
Beltramo  [17],  who  traces  it  back  to  IDA  1974  [2]:  in 
only  one  of  the  17  cases  considered  in  IDA  1974  did  the 
sole-source  supplier  win  the  ensuing  competition. 

Beltramo  accepts  the  IDA  explanation  that  the  sole-source 
producer  had  gotten  trapped  by  his  own  inefficiency. 
Beltramo  offers  no  explanation  of  how  the  producer  was 
trapped,  but  one  mechanism  might  have  been  the  development 
of  inefficient  but  comfortable  ways  of  doing  business. 
Another  mechanism  might  have  been  that  the  sole-source 
producer  felt  he  was  committed  to  the  cost  estimates  by 
which  he  Justified  his  earlier  sole-source  prices.  He 
could  not  drastically  reduce  his  bid  without  Implying  that 
he  had  been  overpricing.  An  alternative  explanation  for 
the  poor  performance  during  competition  of  the  original 
sole  source  would  be  that  the  competition  was  applied 
expressly  because  the  sole-source  producer  was  performing 
poorly. 

An  exception  to  the  exclusive  focus  on  programs  that 
have  experienced  at  least  some  competition  is  found  in 
APRO  80  [26].  This  study  examined  four  sole-source  buys, 
but  only  to  determine  the  rate  of  cost  improvement,  not  as 
a  standard  of  comparison  for  competitive  buys. 

•  Over  half  of  the  53  cases  reviewed  involved  electronic 
components  and  subsystems.  Electronic  components  are 
usually  part  of  a  weapon  system  and  therefore  are  inher¬ 
ently  less  complex  than  the  entire  system.  Also  elec¬ 
tronic  components  may  be  more  like  commercial  products 
than  are  weapon  systems  themselves.  For  these  reasons, 
there  may  be  more  qualified  competitors,  and  thus  more 
effective  competition  and  greater  savings,  for  component 
production  than  for  system  production. 


Beltramo  [17],  quoting  from  a  study  by  ARINC. 


•  More  than  half  of  the  cases  were  winner-take-all  competi¬ 
tions,  which  may  generate  greater  savings  than  split 
buys — a  possibility  discussed  later  under  "Hypotheses." 


•  From  column  3  of  the  table,  it  is  clear  that  many  of  the 
programs  were  active  during  the  period  of  the  large 
Vietnam  buys.  During  that  period,  even  a  split-buy  might 
have  been  efficient  because  output  exceeded  the  minimum 
effective  scale  of  two  plants  (Beltramo  [17],  quoting  a 
study  by  Rucker). 

Variables 


The  variables  used  to  analyze  competition  in  the  literature  have 
tended  to  be  limited  to  those  needed  for  a  "learning  curve":  inflation- 
adjusted  price,  cumulative  quantity,  and  production  rate  (table  2). 

Table  2  also  lists  several  promising  variables  that  rarely  or  never 
appeared  in  the  literature.  The  particular  product  characteristics 
listed  in  the  table  (weight,  speed,  complexity)  would  be  suitable  for  a 
missile  or  airframe.  The  other  variables  are  economic  variables  such  as 
market  structure  (e.g.,  how  many  other  firms  have  produced  this  type  of 
product?),  capacity  utilization  (is  the  industry  particularly  hungry?), 
expectation  of  competition  (was  it  expected  all  along  so  that  the  pros¬ 
pect  of  competition  would  have  an  effect  even  before  the  actuality?), 
and  prices  of  earlier  versions  (was  any  money  being  made  on  previous 
buys  or  previous  versions?). 


TABLE  2 

VARIABLES  USED  IN  ANALYZING  COMPETITION 

Commonly  Used 

Inflation-adjusted  price 
Cumulative  quantity 
Production  rate 
Not  Commonly  Used 

Product  characteristics:  weight,  speed,  complexity,  capacity 
utilization,  and  market  structure 

Expectation  of  competition 

Price  for  earlier  versions 

Profit  rate  on  earlier  buys 

Wages,  materials  prices,  etc. 


These  addicional  variables  have  cross-sectional  variation  as  well 
as  time-series  variation — that  is,  they  vary  across  different  items  as 
well  as  over  time.  The  literature  so  far  has  taken  a  time-series 
approach  almost  exclusively  in  which  data  for  one  program,  considered 
over  time,  are  analyzed  without  reference  to  other  programs.  The  intro¬ 
duction  of  variables  such  as  those  listed  in  table  2  as  “Not  Commonly 
Used"  is  a  step  toward  adding  a  cross-sectional  perspective  to  the 
analysis. 

Screening  Criteria 

The  literature  devotes  much  attention  to  the  question  of  what 
determines  whether  a  program  is  a  promising  candidate  for  competition, 
and,  if  so,  whether  dual  sourcing  is  a  promising  form.  Table  3  lists 
the  factors  discussed  in  the  literature.  The  current  state-of-the-art, 
however,  has  not  gone  beyond  discussion.  For  many  of  the  factors  on  the 
list,  there  appears  to  have  been  no  serious  attempt  at  quantification, 
so  their  Importance  is  unknown.  It  may  be  that  the  factors  explain  much 
of  the  variation  in  savings  from  competition,  or  almost  none.  It  may  be 
that  competition  almost  always  saves  money,  that  it  almost  never  does, 
or  that  these  determinants  make  the  crucial  difference.  The  quantifica¬ 
tion  step  remains  to  be  addressed  in  future  research. 

RESULTS  FROM  THE  LITERATURE 

This  section  describes  hypotheses  in  the  literature  about  the 
effects  of  competition  and  the  evidence  for  them.  It  also  reviews  the 
issues  covered  in  the  literature  with  great  frequency  and  those  covered 
hardly  at  all.  Finally,  it  describes  the  rudiments  of  a  cross-section 
methodology  currently  in  the  literature. 

The  literature  has  focused  mainly  on  calculating  savings  for  indi¬ 
vidual  programs,  not  on  discovering  general  principles  that  would 
involve  hypothesis  specification  and  testing.  Among  the  few  hypotheses 
that  are  stated  explicitly,  the  most  common  is  the  one  mentioned 
earlier — that  the  price  improvement  curve  (PIC)  shifts  and  rotates  when 
competition  begins.  Other  explicit  hypotheses  hold  that  competition 
increases  learning,  and  that  the  prospect  of  competition  itself  can 
lower  price.  Two  other  hypotheses  are  not  stated  directly  but  can  be 
inferred  from  the  literature:  (1)  winner-take-all  competition  creates 
larger  savings  than  split  buys,  and  (2)  results  vary  sharply  both  by 
program  and  by  study. 

There  are  differing  degrees  of  theoretical  reasoning  and  empirical 
evidence  behind  these  hypotheses: 

Hypothesis  1:  Learning  Proceeds  Faster  Under  Competition 

This  hypothesis  has  two  closely  related  interpretations,  both  of 
which  help  distinguish  it  from  the  shif t-and-rotation  hypothesis.  The 
first  interpretation  has  to  do  with  the  prospect  of  competition.  It 


TABLE  3 


HYPOTHETICAL  DETERMINANTS  OF  WHICH  PROGRAMS  SHOULD 
BENEFIT  MOST  FROM  COMPETITION 


_ Determinant _ 

Substantial  production  rate 

Long,  stable  production  period 

Short  lead  time  to  qualify  second  source 

Prime  contractor  make  rather  than  buy 

Short  time  required  to  stabilize  design 

Steep  learning  curve 

Nonrecurring/recurring  cost  low 

No  special  production  skills,  facilities, 
unique  production  processes 

No  proprietary  data 

Absence  of  other  cost-reduction 
incentives 

No  technical  assistance  needed  from 
Initial  source 

Traditional  rivals  for  Initial  source 

Technically  simple 

An  "inefficient"  sole-source  producer 

Low  start-up  costs 

Firm  technical  data  package 

Lack  of  Congressional  resistance  to 
up-front  cos t s 

Slack  industry  demand 


_ Source _ 

GAO  [19],  Trainor  [20] 
Beltramo  and  Jordan  [10] 
Beltramo  [17] 

Trainor,  APR078  [3] 

Trainor,  Beltramo 

Trainor 

Trainor 

Trainor 

Trainor,  Beltramo 

Trainor,  APR078 

Trainor,  Beltramo  and 
Jordan,  APR078 

Trainor 

Beltramo  and  Jordan 

Beltramo  and  Jordan, 
Beltramo 

Beltramo  and  Jordan 

Beltramo 

APR078 

APR078 

APR078 

APR078 


-23- 


holds  Chat  this  prospect  of  competition  changes  the  learning  curve,  even 
before  competition  itself  arrives.  The  second  interpretation,  more 
sweeping  than  the  first,  has  to  do  with  "what  if"  types  of  questions 
(e.g.,  "how  would  things  have  been  different  (before  or  after  competi¬ 
tion)  if  competition  had  been  expected  or  in  force  from  the  outset?") 
Both  interpretations  can  be  explained  with  the  use  of  the  PIC. 

The  first  interpretation,  given  the  prospect  of  competition,  means 
that  the  PIC,  even  before  competition,  will  be  different  from  what  it 
would  have  been  without  the  prospect  of  competition.  This  possibility 
is  illustrated  in  figure  5,  where  the  dotted  lines  suggest  several 
possibilities  had  competition  not  been  in  prospect.  Without  limit 
pricing — that  is,  reducing  prices  in  hopes  of  discouraging  competitors — 
the  initial  prices  would  have  been  higher,  as  shown  by  the  upper  dotted 
line.  Without  skimming — that  is,  raising  initial  prices  to  take  profits 
before  competition  arrives — initial  prices  would  have  been  much  lower, 
as  shown  by  the  lower  dotted  line. 

(lofuritfcmie  seal*) 


CUMULATIVE  QUANTITY 


TIG.  &  OBSERVED  AND  HYPOTHETICAL  PRICE  IMPROVEMENT  CURVES 


Note  that  this  interpretation  contradicts  the  central  hypothesis  of 
the  shift/rotation  literature — that  the  initial  segment  of  the  PIC  is 
independent  of  competition.  This  paper  has  described  ways  in  which  this 
maintained  hypothesis  can  fall — the  skimming  hypothesis,  the  limit¬ 
pricing  hypothesis,  etc. 


The  second,  more  general  interpretation  is  that  no  part  of  the  PIC, 
before  or  after  competition,  is  untouched  by  the  presence  of  competi¬ 
tion,  prospective  or  actual.  To  represent  this  situation,  figure  5 
would  have  dotted  lines  after  competition  as  well  as  before.  This 
interpretation  contradicts  the  shif t/rotation  assumptions  even  more  than 
the  first  interpretation  does. 

There  is  only  scant  evidence  in  the  literature  to  support  either 
interpretation: 

e  With  respect  to  the  first  interpretation,  the  effect  of 
the  prospect  of  competition,  there  is  some  evidence  in  a 
paper  by  Science  Applications  Incorporated  (SAI)  on  the 
AIM-7F  [12].  The  paper  noted  that  the  learning  curve  for 
this  version  of  the  Sparrow  missile  had  a  steeper  slope 
than  earlier  versions.  Since  the  procurement  of  earlier 
versions  was  not  competitive,  the  steeper  slope  is  consis¬ 
tent  with  the  hypothesis  that  competition  steepened  the 
slope.  In  estimating  the  cost  savings  from  competition, 
the  paper  takes  as  the  baseline  rate  of  learning  the  rate 
for  the  AIM-7C,  the  first  production  version  of  the 
Sparrow.  Though  this  evidence  supports  the  hypothesis, 
the  support  is  weak.  The  evidence  pertains  to  only  one 
program,  and  no  alternative  explanations  for  the  steepness 
are  considered. 

e  With  respect  to  the  second  interpretation,  evidence  comes 
from  a  study  by  Scherer  of  World  War  II  bomber  production. 
Scherer  noted  that  the  rate  of  learning  was  greater  with 
competition  than  without  (Scherer  [28,  pp.  120-121]). 

Hypothesis  2:  The  Threat  of  Competition  Can  Be  Effective 

This  hypothesis  is  distinguished  from  the  previous  one  in  that  com¬ 
petition  need  not  actually  materialize.  Nevertheless,  the  evidence, 
which  is  anecdotal,  pertains  only  to  those  cases  in  which  the  threat  was 
genuine.  Consider  one  case,  from  APR079  [23,  p.  58],  For  the  M103 
brass  cartridge  case,  Amron  had  been  the  sole  producer  for  several 
years.  Then,  in  1978,  it  lost  a  winner- take -all  competition  to  National 
Eastern.  Immediately  Amron  offered  a  deep  cut  in  the  price  of  an  option 
on  a  still-current  contract,  from  an  expected  60.72  cents  per  cartridge 
case  to  43.93  cents,  a  drop  of  27.7  percent. 


Hypothesis  3:  Winner-Take- All  Competition  Creates  Larger  Savings  Than 
Split  Buys 

There  are  several  reasons  why  this  hypothesis  seems  plausible  on 
theoretical  grounds.  First,  winner-take-all  does  not  sacrifice  econo¬ 
mies  of  scale  the  way  dual  sourcing  does.  Second,  winner-take-all  puts 
more  pressure  on  the  competitors;  there  is  no  second  place.  Third,  less 


complex  items  are  likely  to  be  those  selected  for  successive  winner- 
take-all  competitions.  Successive  wlnner-take-all  competitions  are 
infeasible  for  complex  items  without  close  commercial  counterparts 
because  the  loser  is  unlikely  to  maintain  idle  production  capacity  until 
the  next  competition.  Fourth,  all  cases  in  this  literature  occurred 
before  the  recent  emphasis  on  split-buy  competition  during  production. 
The  driving  reason  for  split-buy  competition  during  that  earlier  era  may 
not  have  been  expected  cost  savings. 

The  hypothesis  of  greater  savings  for  winner-take-all  has  been  dis¬ 
cussed  several  times  in  the  recent  literature  [5,  10,  11,  17].  Beltramo 
[17,  pp.  79,  82,  109],  in  reviewing  7  split  buys  and  18  winner-take-all 
competitions,  arrived  at  the  following  conclusion  about  this  era: 

"Thus,  all  types  of  competition  do  not  produce  the  same  results: 
Winner-take-all  competitions  usually  result  in  savings;  split-buy  com¬ 
petitions  often  increase  costs." 

A  larger  sample  of  cases  is  depicted  in  figure  6.  The  vertical 
axis  represents  percentage  savings  so  that  the  vertical  scatter  of 
points  shows  the  range  of  results  for  different  programs  with  a  par¬ 
ticular  type  of  competition. 

From  figure  6  and  table  1  it  is  clear  that  the  literature  often 
attributes  substantial  savings  to  competition.  Note,  however,  that  the 
gains  from  competition  seem  to  depend  on  the  type  of  competition — that 
winner-take-all  competitions  generate  larger  savings  than  split  buy/ 
winner-take-all,  and  there  have  been  almost  no  pure  split  buys.  Thus, 
the  distinction  between  types  of  competition  is  important,  and  that  dis¬ 
tinction  has  not  been  emphasized  in  the  literature.  There  is  a  tendency 
to  talk  about  the  gains  from  competition  without  distinguishing  among 
the  different  types. 

Note  too  that  the  results  in  figure  6  refer  to  recurring  costs 
only.  This  does  not  imply  that  the  study  from  which  estimates  were 
taken  is  inferior;  a  number  of  other  studies  also  omit  the  nonrecurring 
costs  of  competition  (such  as  government-supplied  tooling  and  test 
equipment)  as  well  as  other  costs  such  as  the  extra  expense  of  the 
educational  or  directed  buy.  Some  of  the  recent  studies  have  tried  to 
include  such  costs  but  have  found  it  difficult — the  data  are  hard  to 
obtain  and  hard  to  reconstruct  in  retrospect.  What  may  be  needed  is  a 
mechanism  for  systematically  gathering  data  as  it  is  generated  and 
making  it  more  broadly  available  to  the  research  community. 

Future  research  needs  to  focus  in  more  detail  on  nonrecurring 
costs.  Even  a  single  case  study  would  be  useful  in  documenting  how 
estimated  savings  vary  in  response  to  the  way  in  which  these  costs  are 
handled. 
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FIG.  6:  ESTIMATED  SAVINGS  BY  COMPETITION  TYPE 
Sources  —  Type  of  Competition  Beltramo[17] 
—  Estimated  Savings  TASC70[5] 


Until  there  is  such  a  case  study,  only  a  rough  idea  of  the  impor¬ 
tance  of  nonrecurring  costs  can  be  gathered  from  the  current  literature. 
Beltramo  and  Jordan  [10,  p.  10]  estimate  that  incorporating  the  costs  of 
the  directed  or  educational  buy  would  cause  the  following  changes  in 
estimated  savings  from  competition:  TOW,  from  22.6  to  11.0  percent; 
Shillelagh,  from  -6.3  to  -10.4  percent;  and,  as  discussed  above,  AIM-7F, 
from  -20.5  to  -31.4  percent.  These  changes  are  from  4  to  10  percent  of 
total  cost.  In  another  study,  Beltramo  and  Jordan  [12,  p.  12]  project 
that  nonrecurring  costs  may  make  up  about  6  percent  of  recurring  cost 
for  a  buy  of  10,000  AIAAMs. 

Hypothesis  4:  Results  Vary  Substantially  by  Program  and  by  Study 

There  is  already  a  great  deal  of  evidence  in  the  present  literature 
that  results  do  vary  by  program  and  study.  This  evidence  is  summarized 
below.  The  remaining  issue  is  whether  this  variation  will  diminish  over 
time  or  at  least  become  predictable.  Reduction  in  the  variation  by  pro¬ 
gram  is  essentially  a  matter  of  finding  determinants  of  savings  that  can 
predict  the  variation.  Reduction  in  the  variation  by  study  would 
require  increased  agreement  on  assumptions  and  techniques.  Such  reduc¬ 
tion  might  be  accomplished  by  fiat,  but  it  would  be  more  informative  if 
it  were  accomplished  by  testing  the  predictive  ability  of  different 
assumptions  and  techniques.  Such  testing  does  not  appear  to  be  part  of 
the  present  literature. 

Turning  to  the  evidence  of  variation  in  the  present  literature, 
figure  7  illustrates  the  variation  in  results  by  program  and  by  study. 

A  subset  of  programs  was  selected  for  which  there  were  at  least  three 
estimates  of  cost  savings  among  the  five  references  listed  in  the 
figure.  Plotted  on  the  vertical  axis  are  percentage  savings;  the 
horizontal  axis  lists  the  different  programs.  The  variation  in  the  hor¬ 
izontal  dimension,  from  program  to  program,  makes  it  harder  to  predict 
in  advance  how  a  given  competition  will  do. 

Most  of  the  cases  in  the  diagram  showed  estimated  savings  from 
competition.  Nevertheless,  it  should  be  noted  that  none  of  them  were 
pure  dual-source  competition.  All  were  either  winner-take-all  or  dual 
source  followed  by  a  winner-take-all  buyout.  Further,  the  accounting 
for  nonrecurring  cost  was  sketchy.  Thus,  the  point  of  this  diagram  is 
not  that  there  have  or  have  not  been  sizeable  gains  from  competition 
reported  in  the  literature,  but  rather  to  point  out  the  range  of  varia¬ 
tion  in  the  estimates  of  those  gains  in  this  literature.  Indeed,  the 
variation  in  the  vertical  dimension  of  figure  7,  from  study  to  study, 
suggests  that  the  details  of  the  calculations  can  have  a  large  effect. 


One  of  the  best  examples  of  study-to-study  variation  of  estimated 
savings  for  any  given  program  is  the  AIM-7F  program,  which  is  too  recent 
to  have  been  included  in  the  studies  from  which  the  figure  is  drawn. 

The  different  estimates  of  savings  for  the  AIM-7F  are  given  in  table  4. 
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TABLE  4 


ESTIMATED  SAVINGS  FOR  AIM-7F 


_ Source _ 

SAI  (August  1982 )a 

SAI  (September  1982)  : ^ 

Effect  of  educational  or  directed  buy 

SAI  (March  1983)  :c 

Effect  of  competition  on  slope  of 
sole-source  learning  curve 


Estimated  savings 
-20.5% 

-30.4% 

+11.0% 


a.  [ 9 ]  . 

b.  [10]. 

c.  [12]. 


An  SAI  study  in  August  1982  [9]  estimated  that  competition  actually 
resulted  in  a  20.5  percent  increase  in  costs  (a  -20.5  percent  "savings") 
as  shown  in  table  4.  When  the  study  was  updated  for  publication  in 
September  1982,  the  extra  costs  of  the  directed  or  educational  buy  were 
taken  into  account,  Increasing  the  loss  to  over  30  percent.  Then, 
another  assumption  was  changed.  The  prospect  of  competition  was  assumed 
to  have  steepened  the  learning  curve,  even  before  competition  started. 
When  that  assumption  was  incorporated  into  the  analysis,  estimated  sav¬ 
ings  went  from  about  -30  percent  to  about  +11  percent.  So  within  these 
three  studies,  all  by  the  same  authors,  there  was  a  variation  of  about 
40  percent,  depending  on  assumptions. 

Consider  a  different  effect  of  competition  on  the  slope  of  the 
learning  curve.  APR079  [23,  p.  27,  observation  9]  reported  that  the 
percentage  savings  for  the  1976  procurement  of  the  M223  fuze  ranged  from 
6.1  percent  to  27.3  percent  as  the  assumed  rate  of  price  improvement 
ranged  from  10  percent  to  0  percent. 

The  final  SAI  study  discussed  above  noted  even  further  uncertain¬ 
ties  in  results.  For  instance,  how  would  the  long-run  state  of  the  mar¬ 
ket  differ  from  the  present?  One  of  the  companies  producing  the  AIM-7F 
claimed  to  be  losing  money,  suggesting  that  the  savings  might  not  have 
persisted.  Were  savings  genuine?  Did  savings  from  competition  on  one 
program  show  up  as  higher  overhead  costs  allocated  to  another  program? 

There  are  a  number  of  other  reasons  for  variation  in  estimated 
savings  from  study  to  study.  There  are  even  cases  in  which  different 
studies  seem  to  draw  different  numbers  from  the  same  data  set.  Table  5 
summarizes  some  of  the  reasons  for  these  differences. 


TABLE  5 


SOME  REASONS  FOR  DIFFERENCES  ACROSS  STUDIES 


1.  Different  data.  For  example,  studies  differed  on  which  buys  were 
production  (and  therefore  included  in  the  learning  curve)  and  which 
were  R&D  (and  therefore  not  appropriate  for  the  learning  curve). 

2.  Differing  degrees  of  including  nonrecurring  costs,  e.g.,  government- 
paid  tooling  and  test  equipment. 

3.  Differing  degrees  of  including  other  costs,  e.g.,  the  cost  of  the 
directed  or  educational  buy. 

4.  Different  ways  of  estimating  the  hypothetical  rate  of  learning 
during  the  competitive  period: 

a*  Estimated  from  early  production  lots  with  and  without  directed 
buy 

b.  Inferred  from  another  program 

c.  Assumed  from  general  knowledge. 

5.  Different  ways  of  correcting  for  inflation. 

6.  Discounting  or  not. 

7.  Differences  in  assumptions  about  percent  of  program  competed. 

8.  Whether  production  rate  is  included  as  a  determinant  of  price. 

9.  Differences  in  estimating  equations  for  learning  curve. 


ISSUES 

The  literature  on  competition  in  procurement  has  discussed  some 
issues  frequently,  others  hardly  at  all.  As  is  evident  from  table  6, 
the  issues  discussed  often  tend  to  be  learning-curve  type  issues,  such 
as  uncertainty  about  the  rate  of  learning. 

Price  and  Cost 


Little  effort  has  gone  into  distinguishing  between  price  and  cost, 
even  though  reducing  the  difference  is  a  major  way  in  which  competition 
works.  Fee  or  profit  rate  is  a  major  element  of  price  where  the  effects 
of  market  conditions  should  show  up  most  readily.  Interpreting  price  as 
cost  can  create  misleading  estimates  of  the  effect  of  competition.  For 


TABLE  6 


DISCUSSION  OF  ISSUES  IN  THE  LITERATURE 


Discussed  Often 

Nonrecurring  costs: 

—  Start-up  costs 
—  Educational  buy 
—  Purchase  of  data  rights 
—  Creation  of  technical  data  package 
Rate  adjustment 
Effect  of  cumulated  quantity 
Estimation  of  log-log  form 
Shape  of  learning  curve 
Discounting 
Inflation  correction 
Transferability  of  learning  curve 
Discussed  Rarely 

Nonrecurring  costs: 

—  Forfeiture 
—  Expectations 

Extent  of  preexisting  competition 
Standard  of  comparison 
Distinction  price/cost: 

—  Distinction  price  level/rate  of  change 
Market  structure 


Long-run  equilibrium 


instance,  if  producers  were  losing  money  Just  before  competition  and 
responded  by  raising  prices,  it  would  appear  that  the  competition,  not 
the  previous  losses,  caused  the  rise  in  prices.  Here  is  an  actual  case, 
the  M223  fuze,  as  described  in  APR079  [23,  pp.  32-53]. 

“Before  this  acquisition,  Honeywell  was  producing  the 
fuzes  for  a  unit  price  of  $0,512  (FY  1978  dollars) 
and  the  projected  price  was  $0.4977.  After  the 
acquisition  the  weighted  average  price  for  all  six 
producers  was  $0.6225,  giving  an  estimated  loss  of 
25.1  percent. 

Discussions  with  government  contracting  person¬ 
nel  reveal  the  reason  for  this  price  increase.  The 
experienced  contractors,  AVCO,  Dayron,  and  Honeywell, 
had  lost  money  on  their  previous  contracts,  and  they 
now  bid  at  a  more  realistic  level." 

Price  and  Change  in  Price 

Just  as  the  literature  has  failed  to  distinguish  between  price  and 
cost,  it  also  has  failed  to  distinguish  between  price  and  the  change  in 
price.  A  program  whose  price  is  falling  rapidly  is  considered  to  be 
efficient,  without  consideration  of  the  possibility  that  the  reason  for 
the  rapid  fall  in  price  may  be  that  the  price  started  off  unusually 
high.  A  falling  price  does  not  necessarily  mean  that  either  the  final 
price  is  low  or  the  program  is  efficient. 

The  failure  to  distinguish  between  price  and  its  change  shows  up 
most  prominently  in  the  shift/rotation  methodology:  When  the  sole 
source  has  a  steep  learning  curve,  estimated  gains  as  a  result  of  com¬ 
petition  will  tend  to  be  small  or  negative.  But  if  the  steepness  of  the 
curve  actually  results  from  the  unusually  high  initial  price,  then  com¬ 
petition  may  actually  create  large  savings,  which  the  shift/rotation 
methodology  has  no  way  of  uncovering. 

Though  typically  Ignored  in  the  literature,  the  conceptual  distinc¬ 
tion  between  price  and  its  change  is  not  new:  Scherer  [28,  pp.  123-125] 
reviews  research  by  Asher  indicating  that  for  a  sample  of  post-World 
War  II  fighter  aircraft,  those  with  steep  "progress"  (learning)  curves 
tended  to  have  high  values  of  TQ  (the  manhours  per  pound  for  the  first 
unit).  Further,  it  turns  out  that  for  the  actual  quantities,  the  effect 
of  the  steep  slope  was  dominated  by  the  effect  of  Tq  so  that  a  steeper 
progress  curve  actually  indicated  more  total  manhours. 

As  Asher  [27,  p.  83]  himself  states: 


"Since  World  War  II,  the  consensus  of  the  Air  Force 
and  the  airframe  industry  has  been  that  steep  prog¬ 
ress  curves  showing  rapidly  declining  costs  indicate 


efficient  production,  and  that  flat  progress  curves 
showing  slowly  declining  costs  indicate  inefficient 
production.  The  analysis  presented  here,  however, 
suggests  that  such  a  view  may  not  be  warranted.  It 
is  obvious  from  Fig.  4.6,  that  there  are  more  total 
manhours  beneath  a  steep  curve  than  there  are  beneath 
a  flat  curve." 

Others,  too,  have  recognized  the  distinction  between  the  level  of 
price  and  the  rate  of  change  of  price.  IDA79  [4]  includes  a  comparison 
of  the  price  of  military  and  nonmilitary  items  of  the  same  type,  con¬ 
cluding  that  the  military  items  are  no  more  expensive.  Greer  and  Liao 
[16]  evaluate  the  effects  of  competition  on  the  level  of  price  for 
different  missiles.  Both  correctly  use  the  level  of  price  as  the 
variable  for  analysis. 

Other  Issues 


As  shown  in  table  6,  it  should  be  noted  that  even  those  Issues 
listed  as  "discussed  often"  in  the  literature  have  often  not  been  incor¬ 
porated  into  estimates  of  savings.  A  prime  example  is  "nonrecurring 
costs." 


Similarly,  the  issues  listed  in  the  table  as  "discussed  rarely" 
have  not  been  neglected  altogether.  For  example,  Beltramo  and  Jordan 
[12,  p.  34],  implicitly  note  the  issue  of  long-run  equilibrium  when  they 
point  out  that  savings  resulting  from  major  losses  to  the  contractor  are 
not  necessarily  a  cause  for  rejoicing.  The  likely  result  is  fewer  con¬ 
tractors  and  higher  prices  in  the  future.  They  also  note  other  possible 
sources  of  spurious  savings:  loading  of  overhead  onto  other  government 
contracts  and  scrimping  on  engineering  manpower  used  to  develop  future 
missiles. 

Long-Run  Equilibrium 

A  final  issue,  treated  rarely  in  the  literature,  is  the  long-run 
state  of  the  market.  For  instance,  is  it  possible  that  competition  can 
squeeze  a  company  to  greater  efficiency  or  lower  profits  in  the  short 
run,  but  force  it  out  of  business  in  the  long  run?  Similarly,  competi¬ 
tion  may  reduce  price  if  it  surprises  a  producer,  but  in  the  long  run 
such  surprises  will  be  built  into  expectations.  Producers  may  either 
require  long-run  guarantees  or  build  in  an  extra  margin  of  profit  on 
each  program  to  cover  such  contingencies. 


1.  Figure  4.6  displayed  the  family  of  progress  curves  implied  by  Asher's 
estimate  of  the  relation  between  slope  and  intercept.  Steeper  curves 
lay  above  flatter  curves. 
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SUMMARY 


The  research  described  in  this  paper  evaluated  the  literature 
reporting  research  on  estimated  savings  due  to  the  use  of  competition  in 
weapon  system  production.  It  focused  on  the  following  three  aspects  of 
the  literature:  the  analytical  models  and  techniques  used  for  analysis 
of  price  formation  in  sole-source  and  competitive  productions,  the  data 
bases  used  and  reported  for  the  programs  studied,  and  the  estimated 
savings  reported  as  results. 

Models  and  Techniques 

All  of  the  studies  examined  used  the  Price  Improvement  Curve  (PIC) 
model  for  estimating  savings  due  to  competitive  pricing  in  production 
programs  relative  to  estimates  of  prices  of  the  programs  had  they  con¬ 
tinued  as  sole  source.  The  studies  analyzed  each  program  separately  as 
a  time  series  of  production  lots.  The  number  of  lots  for  each  program 
was  usually  10  or  less.  Such  samples  are  far  too  small  to  use  in  making 
any  statements  on  the  statistical  significance  of  price  differences 
before  and  after  competition. 

The  PIC  wa's  found  to  be  too  narrowly  based  on  cumulative  production 
quantity  and  production  rate  to  provide  any  assistance  in  forecasting 
pricing  behavior.  It  tries  to  explain  too  much  with  too  few  variables. 
The  literature  discussed  many  possible  additional  candidate  variables, 
but  only  one  study  [16]  explored  an  alternative  variable,  industrial 
capacity  utilization,  in  its  empirical  work.  CNA’s  research  on  expand¬ 
ing  the  set  of  variables  will  be  reported  in  subsequent  Research 
Memoranda. 

Data  Bases 


A  major  concern  is  representativeness  of  the  data.  The  programs 
used  in  the  reported  studies  might  be  viewed  as  being  from  a  quite  dif¬ 
ferent  era  regarding  the  motivations  for  competition  and  its  use  in 
production,  particularly  dual-source  production.  There  are  potential 
biases  in  both  the  generation  and  selection  of  types  of  competitive  pro¬ 
grams  for  study.  There  may  be  biases  toward: 

•  Using  the  winner-take-all  form  of  competition 

e  Using  winner-take-all  competition  on  programs  that  were 
less  complex  and  were  similar  to  commercial  programs 

e  Using  competition  on  programs  that  were  experiencing 
trouble  as  sole-source  procurements. 

The  winner-take-all  competition  type  had  the  largest  representation  by 
far  of  the  types  presented  in  all  studies.  The  documentation  and  anal¬ 
yses  of  the  data  in  the  literature  were  found  to  be  uneven  in  many 


respects.  Several  studies  had  incomplete  or  no  nonrecurring  cost  data, 
which  biased  estimated  savings  upward.  The  categorization  of  data  by 
types  of  competition  was  inconsistent  across  studies,  particularly 
regarding  dual-source  procurements  and  educational  buys.  The  documenta¬ 
tion  of  data  and  data  sources  was  uneven,  making  replication  of  many  of 
the  analyses  impossible. 

Estimated  Savings  Results 

The  estimated  savings  reported  in  the  literature  should  be  used 
with  great  caution  because  of  the  limitations  of  the  data  bases  and 
analytical  methods  as  summarized  in  the  preceding  sections.  A  reason¬ 
able  generalization  of  the  estimated  savings  results  might  be  as 
follows: 

•  Savings  from  winner-take-all  competitive  procurements  are 
likely,  although  uncertain  and  highly  variable. 

•  No  generalization  on  estimated  savings  from  dual-source 
competitive  procurements  is  possible  based  on  the 
literature  examined. 

Several  programs  reported  in  the  literature  were  dual-source  pro¬ 
curements  for  a  part  of  their  history,  with  a  winner-take-all  procure¬ 
ment  for  the  balance  of  the  program.  The  results  for  the  two  different 
types  of  competition  generally  are  not  reported  separately. 


Further  research  is  required  on  both  methodology  and  data  bases  if 
more  reliable  estimates  of  savings  due  to  competition  are  to  be 
achieved. 
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